Okay, so welcome back everybody to our deep learning lecture and today we want to talk

about object detection and segmentation.

So we are now going towards the applications really of deep learning and two important

applications where the techniques have shown to be very very successful with object detection

and segmentation.

So we will start with a short introduction where we want to really separate the task,

what is the actual task of object detection and segmentation and how they are actually

related to some extent and then we go into the object detection, different methods, then

we go into the segmentation and see how to set this up with fully convolutional networks.

Okay so introduction, well today we want to talk about general image analysis and so far

we have only talked about classifying images to a certain class.

So if you had this kind of images you would say okay this is cat, category cat.

But actually you could also be interested in localizing a certain object or instance

or observation in an image and you could do for example object recognition and localization

and then not separate the instances.

So you would have one box around the object of interest here that would be this box shown

in the image and to be honest this is rarely used because most of the time you are more

interested in identifying the individual objects right.

So you would be talking about object detection and they want to detect different instances

of a certain class and localize it at the same time or in a successive manner as different

approaches to do so.

So this is object detection.

So later in part two we want to talk about segmentation and the first approach that you

can do is semantic segmentation.

So in semantic segmentation you would be interested in identifying all the pixels that belong

to the certain class.

So this would be the segmentation for the class cat.

And what may be even more interesting is semantic instance segmentation where you have a segmentation

and the actual instance identification in the same kind of identification network.

So here you really try to figure out the exact pixels belonging to a certain instance and

the respective class.

So this is an even harder task.

Okay, so let's start with the object detection.

So what we want to do we want to localize and we want to classify.

So one way how you can do that is you try to detect bounding boxes and when you have

the bounding box you can resample them and then apply a classifier.

So first you try to find the snippet of the image that contains probably one instance,

one object and then you classify the object.

So that's a very common approach and mainly the idea is now that you need clever combinations

replacement of these steps to achieve higher speed or accuracy.

And we'll talk about different ways how to achieve that and this is partially also how

things have been developing over the last few years and how they were published.

And you can see that more and more they get advanced and tend to replace all kinds of

parts with neural networks.

So let's see how this goes.

So the first thing is what we're looking for, we're looking for a bounding box and that

is the smallest box by some measure that fully contains the object in question.

So typically you like to use a top left corner, a width and a height and that will tell you

where this object of interest is.

And then you can also put in a classifier confidence in your bounding box detection